1. Field of the Invention
The present invention relates generally to a method and apparatus for producing animations and, more particularly, is directed at producing animations which appear to synchronize with a wide array of rhythmic patterns.
2. Edge Detection in the Human Visual System
Vision researchers have long known that the human visual system includes mechanisms specially adapted to detect edges (i.e. sharp transitions between relatively homogenous regions) in a visual field. The human visual system will sense motion in a visual field when the shape or configuration of edges within the field changes. Experiments demonstrating the edge detecting capabilities of the human visual system are described in numerous scientific publications.
It is also known that the human visual system is capable of detecting edges even in the presence of visual noise. That is, the human visual system may detect edges in a visual field even if the edges are, for example, poorly defined or partially obliterated.
A visual field may, however, contain an edge which is so poorly defined that the edge will not initially be perceived by a human observer. Nonetheless, the same observer may suddenly perceive the edge if, for example, the edge is pointed out to the observer, or if the observer's attention is otherwise focused on the edge. Conversely, such an edge may seem to disappear from the visual field if the observer is distracted or otherwise focused away from the edge. Such edges in a visual field will be referred to as "latent edges."
3. Computer Animation
Modern computer graphics systems typically include a central processing unit (CPU), an image memory which is mapped by hardware locations onto an output display device, and a color map, which provides a correspondence between a given value stored in the image memory and the resulting color and intensity of the corresponding pixel in the output device. Computer graphics systems may be used to create animations by programming the central processing unit to repeatedly change the values stored in the image memory or in the color map, so as to produce a rapidly changing sequence of edges. For example, the central processing unit may be programmed to repeatedly substitute a pre-existing sequence of "frames" of an animation into the image memory. Alternatively, the central processing unit may be programmed to modify the values in the image memory according to a mathematical formula. Rapid changes in the color map may also produce the illusion of dynamic motion in the displayed image.
When animations are created to accompany music or other rhythmic patterns, they are typically hand-crafted. In other words, it is usually necessary for a human operator to painstakingly "key" visual events (corresponding, for example, to the creation of a new edge or repositioning of a set of edges in the displayed image) to rhythmic events (such as crescendos or drum beats). This keying usually involves the creation of new visual material to correspond to particular rhythmic events. In other cases, rather than creating new material, it is possible to alter the time-course of an existing animated sequence to create a tighter synchronization between the sequence and a stream of rhythmic events.
Referring to FIG. 1, manual event keying an animation to, for example, a musical sequence requires a human listener to subjectively interpret an auditory stream 11 (shown as a graph of sound intensity as a function of time) as a series of auditory events 12, and to program a series of visual events 13 which correspond in time with those auditory events 12. As can be seen, there is no obvious way to identify any auditory event such as a drum beat 14, or a harp chord 15, simply by mechanically analyzing the shape of the corresponding auditory stream 11. Thus, to create the series of visual events 13, a human listener must listen to auditory stream 11 and record the relative times of subjective auditory events 12. After these auditory events have been identified, an artist creates a series of visual events 13 at appropriate times to correspond to auditory events 12. When this series of visual events 13 is later displayed in synchronization with auditory stream 11, the observer will perceive visual events occurring in-time to the music. Notice that time-consuming human decisions are necessary at several stages of this process.
Previous attempts to automate this process have primarily focused on creating devices (hardware and/or software) that can automatically parse the auditory stream 11 into a stream of auditory events 12 and thereby eliminate the first tedious step. Such techniques can, however, be quite slow and variable in their results.